Notes on Motion Estimation 1 Geometry: 3d Velocity and 2d Image Velocity

There are a great variety of applications that depend on analyzing the motion in image se quences These include motion detection for surveillance image sequence data compression MPEG image understanding motion based segmentation depth structure from motion obstacle avoid ance image registration and compositing The rst step in processing image sequences is typically image velocity estimation The result is called the optical ow eld a collection of two dimensional velocity vectors one for each small region potentially one for each pixel of the image Image velocities can be measured using correlation or block matching for example see Anan dan in which each small patch of the image at one time is compared with nearby patches in the next frame Feature extraction and matching is another way to measure the ow eld for reviews of feature tracking methods see Barron or Aggarwal and Nandhakumar Gradient based algorithms are a third approach to measuring ow elds for example Horn and Schunk Lucas and Kanade Nagel A fourth approach using spatiotemporal l tering has also been proposed for example Watson and Ahumada Heeger Grzywacz and Yuille Fleet and Jepson This handout concentrates on the lter based and gradient based methods Emphasis is placed on the importance of multiscale coarse to ne re ne ment of the velocity estimates For a good overview and comparison of di erent ow methods see Barron et al